METHOD AND SYSTEM FOR PROCESSING A THREE- 
. DIMENSIONAL SHAPE DATA 

This application is based on Japanese Patent Application 
No. 372356/1 999 filed on December 28, 1999, the contents of 
which are hereby incorporated by reference. 



BACKGROUND OF THE INVENTION 
Field of the Invention 
10 The present invention relates to a method and a system 

C3 for processing a three-dimensional shape data. 

%! Description of Prior Art 

b \j Conventionally, modeling which utilizes a three- 

1*1 dimensional measuring apparatus is performed. For example, a 

s F 15 three-dimensional measuring apparatus using a light-section 

M; method, scans optically an object by projecting and deflecting a 

Til 

.ifV slit light and calculates a three-dimensional shape data by 

applying a principle of a triangular surveying. Such 
t3 measurement is performed around the object, a plurality of the 

20 obtained three-dimensional shape data are connected together, 
so as to obtain the shape model for the object as a whole. 

Because of the measuring conditions, such as shape or 
direction of an object, or lighting, the three-dimensional shape 
data sometimes cannot be obtained perfectly, the lack of the data 
25 may occur. In this case, the filling of the lack part, namely the 
data correction of the three-dimensional shape data is 
performed. 

Conventionally, as a method for filling the portion with 
lack of data, the method of using a grid as a fundamental model, 
30 generating a new curved surface by transforming the grid, and 
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connecting the generated surface with the outline of the portion 
«. with lack of data is provided. The unexamined Japanese patent 
publication No. 1 1-15994 discloses a computer system in which 
an operator can adju'st the'curved surface by trial and error with 
monitoring the result so as to make the filling natural. In this 
system, by designating at least one point inside the curved 
surface and a target position, the curved surface is automatically 
modified to be smooth surface including the target position, 
automatically, and the numerical data representing the change of 
the shape is displayed. The operator evaluates the modification 
by the numerical data, if the operator is not satisfied with the 
result, then he or she designates again a point inside the curved 
surface and a target position. 

Conventionally, there is a problem that the operator must 
designate a large number of values for adjusting the curved 
surface to be added to the shape model and the data correction 
operation of the shape model is troublesome. Especially, if the 
shape model has a complex surface with ups and downs, the 
operator must designate many points, therefore the load on the 
operator is heavy. 

SUMMARY OF THE INVENTION 
The object of the present invention is to provide a 
method and system for adding a required curved surface to a 
portion of the shape model by a simple and easy input operation. 

According to an aspect of the present invention, a method 
comprises the steps of obtaining three-dimensional shape data 
representing a three-dimensional shape model, designating a 
portion of the three-dimensional shape model required to be 
corrected, displaying a surface to be joined to the designated 



portion, modifying a shape of the surface corresponding to an 
i . alteration of a parameter, and re-displaying the modified surface 

in response to the reception of alteration. 

According to 'anotfrer aspect of the present invention, a 
5 method comprises the steps of displaying a three-dimensional 
shape model and a surface to be joined to the three-dimensional 
shape model, the shape of the surface is defined by at least one 
parameter, obtaining only one value of the parameter, and 
modifying the displayed surface based on the obtained value of 
10 the parameter. 

The other objects and features of the present invention 
v3 will be more fully understood from the following detailed 

J description with reference to the accompanying drawings. 

ffT; 

fU 15 BRIEF DESCRIPTION OF THE DRAWINGS 

3 Fig. 1 is a block diagram showing a three-dimensional 

?U processing apparatus according to an embodiment of the present 

invention; 

£3 Fig. 2 is a schematic diagram for illustrating a filling 

20 process; 

Fig. 3A is a diagram for showing a form of a dialogue of 
a first operation example; 

Fig. 3B is a diagram for illustrating a changing of a 
displayed monitor image of the first operation example; 
25 Fig. 4A is a diagram for showing a form of a dialogue of 

a second operation example; 

Fig. 4B is a diagram for illustrating a changing of a 
displayed monitor image of the second operation example; and 
fa ^ ri*g. § is a diagram for illustrating a data processing for 

30 filling of a lack portion. 
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DETAILED DESCRIPTION OF THE INVENTION 

i 

Fig. 1 is a block diagram showing a configuration of a 
three-dimensional processing system of an embodiment 
5 according to the present invention. 

Referring to Fig. 1 5 the three-dimensional processing 
system 1 is a computer system which comprises a bus 10, a CPU 
(central processing unit) 11, a memory 12, an auxiliary storage 
device 13, an input/output interface 14, a keyboard 15, a mouse 

10 16, and a display 17. 

The CPU 1 1 executes various processes including a data 
modification according to the present invention. The memory 
12 includes a ROM 12a for storing a control program in which a 
sequence executed by the CPU 11 is scripted, and a RAM 12b 

15 providing a data storage area and a working area required for the 
execution of the various processes by the CPU 11. The 
auxiliary storage device 13 is used for storing an objective data 
such as a three-dimensional data (shape data), a two-dimensional 
color image data, and so forth. The input/output interface 14 is 

20 used for inputting an objective data from the external of the 

device or for outputting a generated data to the external of the 
device. 

The keyboard 15 and the mouse 16 are used for inputting 
various instructions or settings. The display 17 displays 
25 images of three-dimensional data and the generated surface, and 
also is used for displaying a processing status, a processed 
result, and a progressing status. The display 17 can display a 
plurality of windows by a multi-window system. 

By using the three-dimensional processing system 1, a 
30 required curved surface data can be added to a three-dimensional 
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data which is either inputted from the three-dimensional 
« measuring apparatus (not shown), or generated by the modeling 
process. This function is suitable for a full-filled process of a 
lack portion (hole) of a sh'ape model. 
5 Fig. 2 is a schematic diagram of a filling process. 

The three-dimensional data as an object for correction 
shown in the example is a shape model M of a mascot doll. The 
lack portion 90 exists in the shape model M. Namely, the 
mascot doll M lacks the upper side surface portion. The- 
10 filling-a-hole process is a data processing such that a grid-like 
fs% plane surface m is attached on a lack portion 90, the plane 

^ surface m is modified to an appropriate curved surface 

=p corresponding to the shape model M, so as to join the curved 

fn surface to the outline of the lack portion. Thus, with the three- 

's 15 dimensional processing systeml, an operator (user) can fill the 

; lack portion with the desired curved surface by a simple and easy 

fU operation, 
frp [First operation example] 

Jif Figs. 3A and 3B are diagrams showing a first operation 

20 example. Fig. 3A shows a form of a dialogue window and Fig. 
3B shows a changing of an image on a monitor display. 

An operation sequence of the first example is as follows: 
(1) In the modeling mode capable of correcting the shape 
model M, an operator instructs to execute filling a hole with 
25 designation of the area which contains a lack portion 90. Thus, 
the three-dimensional processing system 1 specifies an object of 
a process by detecting an outline of the lack portion 90, and 
displays a dialogue window 71 for querying a parameter value 
(coefficient y , mentioned later) onto a screen of a display 17. 
30 (2) The operator fixes (designates) a parameter value 
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between a range of 0-1 by dragging a knob 714 of a slider in the 
■ dialogue window 71. Then, the operator clicks an "Apply" 
button 711. The three-dimensional processing system 1 
performs a calculation for'filling a hole, and displays an image 
5 G4 showing a result of filling on a monitor area 75 in the screen. 
This calculation process corresponds to a first step of the 
present invention and displaying of the result corresponds to a 
second step. The image G4 shows a condition that the curved 
surface m4 is positioned on the shape model M. A curvature of 

10 the curved surface m4 is defined by the parameter value. 

(3) When a sufficient result could not be obtained, the 
operator resets appropriately the parameter value, and clicks the 
"Apply" button 711 again. The operator repeats this operation 
until the sufficient shape is obtained. In response to the 

15 modification of the parameter value, the three-dimensional 

processing system 1 recalculates and displays the result. This 
process corresponds to a third step of the present invention. In 
the example shown in Fig. 3B, after displaying the image G4, the 
parameter is modified twice in total, the displayed image is 

20 changed from the image G4 to an image G2, and further from the 
image G2 to an image G3, according to the modification of the 
parameter. Thus, by making the parameter value larger than 
that of the curved surface m4, the curved surface m2 having a 
more gentle curve than that of the surface m4 is applied, and by 

25 selecting the parameter value between the surface m4 and the 
surface m2, the curved surface m3 having a curve more gentle 
than the surface m4 and sharper than the surface m2 is applied. 

Each of images G2 3 G3, and G4 is obtained by projecting 
the shape model located in the virtual space onto the screen. 

30 The operator can select a display mode such as a surface model 
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to which a shading is applied, a wire frame model, and the like. 

(4) If the sufficient result is obtained, an "OK" button 
712 is clicked. Thus, the filling-a-hole process is completed. 
The shape model filled is stored in the auxiliary storage device 
5 13 as the result corrected three-dimensional data. 
[Second operation example] 

Figs. 4A and 4B are diagrams showing a second operation 
example. Fig. 4A shows a form of a dialogue window and Fig. 
4B shows a changing of an image on a monitor display. 
10 An operation sequence of the second example is as 

follows : 

v3 (1) In the modeling mode, an operator instructs to 

execute filling a hole with the designation of the area which 

^ contains a lack portion 90. Thus, the three-dimensional 

m 

fU 15 processing system 1 specifies an object of a process by detecting 

s " an outline of the lack portion 90, and calculates for a filling 

process by applying a default value of the parameter (e.g., 0.5). 
^ The three-dimensional processing system 1 displays an image G3 

C3 showing a result of the filling process on a monitor area 75 in 

20 the screen and displays a dialogue window 72 for querying 

whether a parameter value is appropriate. 

(2) The operator can fix a parameter value between a 
range of 0-1 by dragging a knob 724 of a slider in the dialogue 
window 72. In response to dragging, the calculation for filling 

25 a hole is performed by applying the parameter value of the 

dragged portion, and the result images Gl-5 are displayed in 
real time. Thus, only by operating the slider, the shape of a 
filled portion can be changed freely either to be flat or sharp. 

(3) If the sufficient result is obtained, an "OK" button 
30 722 is clicked. Thus, the filling-a-hole process is completed. 
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With such a second operation, the operation of the filling 
* process can more rapidly proceed than the first operation. 
However, this operation requires that the process performances 
of the devices including CPUll are sufficiently high. 
5 Hereinafter described the calculation for the filling 

process. 

As shown in Fig. 5A, it is considered that an x-y plane 
along with the lack portion (hereinafter refers to as a "hole") 90 
and z direction perpendicular to the x-y plane is set as the 

10 direction for attachment, plane (grid) m which is larger than the 
hole 90 is prepared and applied to the hole 90. Moving the grid 
points inside the hole 90 in the z direction generates the curved 
surface. When positioning of each grid point of the plane m, an 
influence of the position (inclination) of the data of the 

15 periphery of the hole 90 in the shape model M is considered. 

Therefore, the grid points are calculated to minimize a sum up to 
the second order differentiation among the interpolation points 
on the boundary condition of data position of a periphery of the 
hole. Thus, by changing coefficients of terms of a first order 

20 differential and a second order differential, the shape of the 

generated curved surface can be changed, and the filling process 
according to the intention of the user can be achieved. 

As shown with a wide frame line in Figs. 5A and 5B, a 
bounding box BB is generated on the hole 90. The grid is 

25 formed within the bounding box BB and also outside of the 

bounding box BB the grid is formed by 1 line around it. The 
number of the grid point is automatically calculated from the 
three-dimensional data on the periphery of the hole 90. The 
user can also designate the number of the grid points. 

30 The curved surface is generated by calculating the 
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position of the grid points inside the hole 90 (white circle O 
of Figs. 5A and 5B) in the z direction. Those grid points are 
unknown data. The grid points which exist inside the 
bounding box BB and outside the hole 90 (dark circle # of Figs. 
5 5A and 5B) are known data, and those are used as boundary 

values for connecting the shape model M with the curved surface. 
The grid points outside the hole 90 on the bounding box BB 
(dark square ■ of Figs. 5A and 5B) are boundary values. 
These boundary values can be obtained by projecting the grid 
10 points to a polygon of the shape model M. 

These boundary values, known data, and unknown data 
are two-dimensional array as u y , x , y = 0, ... Ny + 3, x = 0, ... Nx + 3 
(where Nx and Ny are the number of the grid points in the x 
direction and y direction, respectively, inside the bounding box 
15 BB). The boundary values and the known data are already 

obtained. The grid intervals in the x direction and y direction 
are Hx and Hy, respectively. 

To obtain unknown data, an energy function is 
determined and the unknown data can be determined to minimize 
20 the energy function. 

E(u) = S(u) + P(u) 
where S(u) represents the degree of less smoothness, and P(u) 
represents a gap from the restriction condition. 

S(u) is represented by the sum of the first order 
25 differentiation and the second order differentiation, and P(u) is 
not considered. 

Therefore: 
(Expression 1 ) 

1 f Ny+2 Nx+2 v 9 . v _ 2 _ 

Ny+2 Nx+2 0 9 2 1 

+ 2 + 2 «/ + «/]} 

i J J 

P(u)= 0 



• # 

In S(u), if the coefficient r of the first order 
• differential term is near 1 5 the influence of the first order 
differential term becomes remarkable. Therefore, the curvature 
of the generated curved surface becomes small and flat. On the 
contrary, if the coefficient r of the first order differential 
term is near 0, the influence of the second order differential 
term becomes remarkable. Therefore, the curvature of the 
generated curved surface becomes large. 

Then, the u 0 is calculated to minimize the energy 
function E(u). If u = uo, minimum E(u) can be obtained, 
V E(u 0 ) = V S(u 0 ) = 0. 

By expanding this, the primary simultaneous equation 
can be obtained. The unknown data is derived as a solution of 
the primary simultaneous equation. The curved surface can be 
generated by the derived unknown data. By modifying the 
value of r 3 filling the user's desires can be achieved. By 
generating the curved surface using only unknown data, and 
connecting the generated surface with the outline of the hole 90, 
then the filling process is completed. 

In the embodiments described above, the parameter 
defining the curvature corresponding to one input item can be 
plural (for example, independent parameters are used in the x 
axis direction and y axis direction). The graphical user 
interface for modifying the operation of the parameter is not 
limited to slider and thus a dial-like member can be used. 

According to the embodiments described above, desired 
curved surface can be added to a part of the shape model by the 
simple and easy operation. 
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